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The  results  of  this  grant  cover  a  wide  range  of  issues  in  knowledge  representation,  natural 
language  and  learning.  The  early  years  of  the  grant  primarily  center  on  the  development  of 
connectionist  approach  to  representation  and  language.  At  the  same  time,  the  development  of  plan- 
based  models  of  dialog  by  the  PI  was  setting  the  ground  for  a  major  push  in  the  area  of  natural 
language  dialog.  The  principal  objective  of  the  dialog  project  was  to  develop  a  system  for 
representing  and  reasoning  about  the  discourse  context  in  extended  man-machine  dialogs.  The 
focus  areas  included  the  development  of  a  model  that  captures  the  structure  of  natural-language 
dialog,  the  development  of  a  general  knowledge  representation  for  capturing  a  wide  range  of 
natural  language  semantics,  and  the  development  of  a  general,  error-tolerant  parser  and  semantic 
interpreter  for  English  that  can  be  guided  by  discourse  information.  Specifically,  we  developed  a 
model  of  discourse  plans  that  includes  actions  such  as  introducing  a  new  topic,  as  well  as  the 
actions  of  clarifying,  correcting  or  acknowledging  parts  of  the  previous  dialog.  We  also  explored 
how  the  “traditional”  language  components,  i.e.  parsing,  semantic  interpretation  and  discourse 
processing,  could  be  adapted  to  the  dialog  scenarios. 

The  best  description  of  the  specific  results  of  this  project  are  found  in  the  publications  of  the 
groups  while  under  this  grant  This  report  presents  these  publications  together  with  a  brief 
summary  of  the  importance  of  each  aspect  of  the  project 
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1.  Connectionist  Models 

At  the  beginning  of  the  80’s,  almost  no  work  was  being  pursued  in  the  areas  now  referred  to  as 
connectionist  models  and  neural  networks.  In  fact,  little  had  been  done  since  the  early  work  on 
Perceptrons,  and  the  negative  results  of  that  project  were  believed  to  show  that  the  approach  was 
impractical.  Jerome  Feldman,  in  the  early  ‘80’s,  disputed  this  claim  and  in  a  series  of  landmark 
papers  introduced  a  version  of  connectionist  models  and  showed  that  the  approach  has  some 
significant  advantages  for  certain  tasks  over  conventional  approaches.  The  following  papers 
include  some  of  the  early  papers  that  defined  and  defended  the  approach  and  then  a  significant 
body  of  literature  exploring  connectionist  models  in  various  different  application  areas.  Significant 
results  were  obtained  in  the  application  areas  of  computer  vision,  knowledge  representation,  and 
natural  language  parsing. 
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Cottrell,  G.W.,  “A  model  of  lexical  access  of  ambiguous  words,”  Proc.,  Nat’l.  Conf.  on  Artificial 
Intelligence ,  61-67,  Austin,  TX,  August  1984. 

Cottrell,  G.W.,  “Parallelism  in  inheritance  hierarchies  with  exceptions,”  Proc.,  Workshop  on 
Non-Monotonic  Reasoning ,  New  Paltz,  NY,  Ocober  1984;  Proc.,  9th  Int’l.  Joint  Conf.  on 
Artificial  Intelligence,  194-202,  Los  Angeles,  CA,  August  1985. 

Cottrell,  G.W.  and  S.L.  Small,  “A  connectionist  scheme  for  modeling  word  sense 
disambiguation,”  Cognition  and  Brain  Theory  6 , 1,  89-120, 1983. 

Cottrell,  G.W.  and  S.L.  Small,  “Viewing  parsing  as  word  sense  discrimination:  A  connectionist 
approach,”  in  G.  Guida  and  B.  Bara  (Eds.).  Natural  Language  Processing.  North  Holland,  1984. 

Fanty,  M,  “A  connectionist  simulator  for  the  Butterfly,”  TR  164,  Computer  Science  Dept,  U. 
Rochester,  January  1986. 

Fanty,  M.A.,  “Context-free  parsing  with  connectionist  networks,”  Proc.,  Neural  Networks  for 
Computing  Conf,  Snowbird,  UT,  April  1986. 

Feldman,  J.A.,  “Dynamic  connections  in  neural  networks,”  Biological  Cybernetics  46,  27-39, 
1982. 

Feldman,  J.A.,  “Brains  and  robots,  two  weak  attempts  at  the  grand  synthesis,”  review.  Cognition 
and  Brain  Theory,  1984. 

Feldman,  J.A.,  “Review  of  The  Architecture  of  Cognition  by  John  R.  Anderson,”  Cognitive  and 
Brain  Theory,  1984. 

Feldman,  J.A.,  “Massive  parallelism  in  natural  and  artificial  intelligence  ”  BYTE,  April  1985. 

Feldman,  J.A.,  “Four  frames  suffice:  A  provisional  model  of  vision  and  space,”  Behavioral  and 
Brain  Sciences  8, 265-289,  June  1985. 

Feldman,  J.A.,  “Energy  and  the  behavior  of  connectionist  models,”  TR  155,  Computer  Science 
Dept,  U.  Rochester,  November  1985. 

Feldman,  J.A.,  “Connectionist  models  and  parallelism  in  high  level  vision,”  CVGIP  31  (Special 
Issue  on  Human  and  Machine  Vision),  178-200, 1985. 

Feldman,  J.A.,  “Connectionist  representation  of  concepts,”  in  D.  Waltz  and  J.A.  Feldman  (Eds.). 
Connectionist  Models  and  their  Applications.  Norwood,  NJ:  Ablex  Publishing  Company,  1987. 
Also  in  Proc.,  Conf.  on  Connectionism  in  Perspective,  U.  Zurich,  Switzerland,  October  1988. 

Feldman,  J.A.,  “Energy  methods  in  connectionist  modelling,”  in  P.A.  Devijver  and  J.  Kittler 
(Eds.).  Pattern  Recognition,  Theory  and  Applications.  NATO  ASI  Series  in  Computer  Science, 
Vol.  F30.  Berlin:  Springer  Verlag,  1987. 

Feldman,  J.A.,  “A  functional  model  of  vision  and  space,”  in  M.  Arbib  and  A.  Hanson  (Eds.). 
Vision,  Brain  and  Cooperative  Computation.  Cambridge,  MA:  MIT  Press/Bradford  Books,  1987. 

Feldman,  J.A.,  ‘Time,  space  and  form  in  vision,”  TR  244,  Computer  Science  Dept.,  U. 
Rochester,  November  1988. 

Feldman,  J.A.,  “Structured  neural  networks  in  nature  and  in  computer  science,”  in  R.  Eckmiller 
and  C.  von  der  Malsburg  (Eds.).  Neural  Computers.  NATO  ASI  Series,  Series  F:  Computer  and 
Systems  Sciences  Vol.  41  {Proc.,  NATO  Advanced  Research  Workshop  on  Neural  Computers, 
Neuss,  Germany,  1987).  Berlin:  Springer  Verlag,  1988. 

Feldman,  J.A.,  “Neural  representation  of  conceptual  knowledge,”  TR  189,  Computer  Science 
Dept,  U.  Rochester,  June  1986;  in  L.  Nadel,  L.A.  Cooper,  P.  CuUcover,  and  R.  Michael  Hamish 
(Eds.).  Neural  Connections  and  Mental  Computation.  Cambridge,  MA:  Bradford  Books/MIT 
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Press,  1989.  Also  in  N.  Sharkey  (Ed.).  Models  of  Cognition.  Norwood,  NJ:  Ablex  Publishing 
Corp.,  1989. 

Feldman,  J.A.  and  D.H.  Ballard,  “Computing  with  connections,”  to  appear  in  A.  Rosenfeld  and  J. 
Beck.  Human  and  Machine  Vision.  Academic  Press,  1983. 

Feldman,  J.A.  and  D.H.  Ballard,  “Connectionist  models  and  their  properties,”  Cognitive  Science 
6,  205-254,  1982;  also  in  D.  Waltz  and  J.A.  Feldman  (Eds.).  Connectionist  Models  and  their 
Applications.  Norwood,  NJ:  Ablex  Publishing  Company,  13-62, 1988. 

Feldman,  J.A.,  M.A.  Fanty,  N.  Goddard,  and  K.J.  Lynne,  “Computing  with  structured 
connectionist  networks,”  TR  213,  Computer  Science  Dept,  U.  Rochester,  April  1987;  invited 
paper,  Commun.  of  the  ACM  31,  2,  170-187,  February  1988. 

Feldman,  J.A.,  M.A.  Fanty,  and  N.  Goddard,  “Computing  with  structured  neural  networks,” 
IEEE  Computer  21, 3, 91-103, 1988. 

Feldman,  J.A.  and  L.  Shastri,  “Evidential  reasoning  in  semantic  networks,”  Proc.,  9th  Int'L  Joint 
Conf.  on  Artificial  Intelligence,  465-474,  Los  Angeles,  CA,  August  1985. 

Feldman,  J.A.  and  L.  Shastri,  “Neural  nets,  routines  and  semantic  networks,”  Directions  in 
Cognitive  Science,  1986. 

Goddard,  N.H.,  “Representation  and  recognition  of  biological  motion,”  Proc.,  Cognitive  Science 
Conf,  August  1988. 

Goddard,  N.H.,  “The  interpretation  of  visual  motion:  Recognizing  moving  light  displays,”  IEEE 
Workshop  on  Visual  Motion,  Irvine,  CA,  April  1989. 

Hollbach  (Weber),  S.C.,  “Direct  inferences  in  a  connectionist  knowledge  structure,”  in  TR  209 
(Proc.,  1988  Open  House),  Computer  Science  Dept,  U.  Rochester,  May  1988;  Proc.,  10th 
Annual  Meeting,  Cognitive  Science  Society,  608-614,  Montreal,  Canada,  August  1988. 

Porat,  S.,  “Stability  and  looping  in  connectionist  models  with  asymmetric  weights,”  TR  210, 
Computer  Science  Dept,  U.  Rochester,  March  1987. 

Porat,  S.,  “Infinite  behavior  in  connectionist  models  with  asymmetric  weights,”  1987-88 
Computer  Science  and  Engineering  Research  Review,  Computer  Science  Dept,  U.  Rochester, 
October  1987;  revised  version  in  R.  Trappl  (Ed.).  Cybernetics  and  Systems  '88  (Proc.,  9th 
European  Meeting  on  Cybernetics  and  Systems  Research).  Dordrecht:  Kluwer  Academic 
Publishers,  1023-1030,  1988. 

Porat,  S.  and  J.A.  Feldman,  “Learning  automata  from  ordered  examples,”  TR  241,  Computer 
Science  Dept,  U.  Rochester,  April  1988. 

Shastri,  L.,  “Evidential  reasoning  in  semantic  networks:  a  formal  theory  and  its  parallel 
implementation,”  Ph.D.  Thesis  and  TR  166,  Computer  Science  Dept,  U.  Rochester,  September 
1985. 

Shastri,  L.  and  J.A.  Feldman,  “Evidential  reasoning  in  semantic  networks:  A  formal  theory,” 
Proc.,  9th  Int'L  Joint  Conf.  on  Artificial  Intelligence,  465-474,  August  1985. 

Shastri,  L.  and  J.A.  Feldman,  “Neural  nets,  routines  and  semantic  networks,”  in  N.  Sharkey 
(Ed.).  Advances  in  Cognitive  Science.  Ellis  Horwood  Publishers,  1986. 
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2.  Plan-based  Models  of  Dialog 

Since  being  introduced  by  Allen,  Cohen  and  Perrault  in  the  early  80’s,  plan-based  models  of 
speech  acts  have  had  a  strong  impact  on  all  work  in  natural  language  dialog  systems.  Throughout 
this  grant  period,  this  approach  was  extended  in  significant  ways  to  handle  extended  dialogs  in 
addition  to  individual  speech  acts.  In  particular,  Allen  and  Litman  introduced  the  method  of  multi¬ 
level  plan  recognition,  where  plans  were  used  to  represent  both  the  domain  information  and  the 
discourse  structure,  and  showed  how  this  model  could  handle  clarification  subdialogs.  The  multi¬ 
level  plan-based  representations  are  now  used  in  most  natural  language  dialog  systems  in  the 
research  community,  and  are  starting  to  be  explored  for  dialog  management  in  applied  dialog 
systems. 

While  the  plan-based  model  of  speech  acts  is  very  attractive  from  a  theoretical  standpoint  and 
has  wide  coverage  of  the  behavior  observed,  it  has  proved  to  be  inefficient  for  use  in  practical 
systems.  Allen  and  Hinkelman  developed  a  method  that  allows  the  use  of  a  simple  structural 
approach  for  most  speech  acts  without  sacrificing  the  generality  of  the  method.  This  technique  is 
now  being  used  in  several  dialog  projects  under  development 

Allen,  J.F.,  “ARGOT:  A  system  overview,”  Int’l.  J.  of  Computers  and  Mathematics  9  (Special 
Issue  on  Computational  Linguistics),  1, 1983. 

Allen,  J.F.,  “Recognizing  intentions  from  natural  language  utterances,”  in  M.  Brady  (Ed.). 
Computational  Models  of  Discourse.  MTT  Press,  1983. 

Allen,  J.F.,  “Discourse  structure  in  the  TRAINS  project”  Proc.,  DARPA  Speech  and  Natural 
Language  Workshop ,  Pacific  Grove,  CA,  February  1991. 

Allen,  J.F.,  “The  TRAINS  project,”  Proc.,  4th  DARPA  Workshop  on  Speech  and  Natural 
Language,  Morgan  Kaufmann  Publishers,  February  1991. 

Allen,  J.F.,  S.  Guez.  L.J.  Hoebel,  E.A.  Hinkelman,  K.J.  Jackson,  A.I.  Kyburg,  and  D.R. 
Traum,  “The  discourse  system  project,”  TR  317,  Computer  Science  Dept,  U.  Rochester, 
November  1989. 

Allen,  J.F.  and  E.A.  Hinkelman,  “Using  structural  constraints  for  speech  act  interpretation,” 
Proc.,  DARPA  Speech  and  Natural  Language  Workshop,  Morgan  Kaufmann,  Publishers,  Fall 
1989. 

Allen,  J.F.  and  D.J.  Litman,  “Plans,  goals  and  natural  language,”  Proc.  of  the  IEEE  74  (Special 
Issue  on  Natural  Language  Processing),  7,  July  1986;  1986-87  Computer  Science  and  Engineering 
Research  Review,  Computer  Science  Dept,  U.  Rochester,  September  1986. 

Allen,  J.F.  and  L.K.  Schubert,  “The  TRAINS  project”  TR  382  and  TRAINS  TN  91-1,  Computer 
Science  Dept.,  U.  Rochester,  May  1991. 

Hinkelman,  E.  A.,  “Relevance:  Computation  and  coherence,”  commentary.  Behavioral  and  Brain 
Sciences  10,  4,  December  1987. 

Hinkelmen,  E.A.,  “Plans,  speech  acts,  and  conversation  implicature,”  in  TR  209  {Proc.,  1988 
Open  House),  Computer  Science  Dept,  U.  Rochester,  May  1988. 

Hinkelmen,  E.A.,  “Linguistic  and  pragmatic  constraints  on  utterance  interpretation,”  Ph.D.  Thesis 
and  TR  288,  Computer  Science  Dept,  U.  Rochester,  May  1990. 
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Hinkelman,  E.A.  and  J.F.  Allen,  ‘Two  constraints  on  speech  act  ambiguity,”  Proc.,  Annual 
Conf.,  Association  for  Computational  Linguistics,  Vancouver,  B.C.,  June  1989;  expanded  version 
appeared  as  TR  271,  Computer  Science  Dept.,  U.  Rochester,  April  1989. 

Litman,  D.J.  and  J.F.  Allen,  “A  plan  recognition  model  for  subdialogues  in  conversation,”  TR 
141,  Computer  Science  Dept,  U.  Rochester,  November  1984. 

Litman,  D.J.  and  J.F.  Allen,  “A  plan  recognition  model  for  clarification  subdialogues,”  Cognitive 
Science  11,  1987. 

Litman,  D.J.  and  J.F.  Allen,  “Discourse  processing  and  commonsense  plans,”  in  P.R.  Cohen,  J. 
Morgan,  and  M.E.  Pollack  (Eds.).  Intentions  in  Communication.  Cambridge,  MA:  Bradford 
Books  /  MIT  Press,  1990. 

Martin,  N.G.,  J.F.  Allen,  and  C.M.  Brown,  “ARMTRAK:  A  domain  for  the  unified  study  of 
natural  language,  planning,  and  active  vision,”  TR  324,  Computer  Science  Dept,  U.  Rochester, 
January  1990. 

Martin,  N.G.  and  B.W.  Miller,  “The  TRAINS-90  simulator,”  TRAINS  TN  91-4,  Computer  Science 
Dept.,  U.  Rochester,  May  1991. 

Traum,  D.R.,  “The  discourse  reasoner  in  TRAINS-90,”  TRAINS  TN  91-5,  Computer  Science 
Dept,  U.  Rochester,  May  1991. 


3.  Natural  Language  Semantics 

One  of  the  crucial  problems  preventing  the  development  of  general  natural  language 
understanding  systems  is  the  lack  of  a  domain-independent  model  of  semantics.  Such  a  model 
would  have  to  delicately  balance  the  need  for  an  effective  reasoning  system  (and  hence  an 
expressively  limited  logical  language),  versus  the  need  for  simple,  compositional  semantic 
interpretation  (hence  the  need  for  a  richly  expressive  language  mirroring  the  structure  of  natural 
language).  The  problems  facing  the  resolution  of  this  problem  are  described  in  detail  in  Allen 
(1991)  below.  Len  Schubert  has  made  significant  progress  on  this  problem  with  the  development 
of  his  episodic  logic.  While  retaining  a  first-order  model  theory,  he  has  developed  a  language  that 
captures  many  of  the  structural  properties  of  natural  language.  As  a  result,  the  semantic 
interpretation  process  is  relatively  simple,  yet  the  representation  supports  effective  inference 
procedures.  This  work  is  described  in  the  papers  listed  below,  including  the  one  that  won  the  best 
paper  prize  at  the  first  International  Conference  on  Principles  of  Knowledge  Representation  and 
Reasoning  in  1989. 

Allen,  J.F.,  “Natural  language,  knowledge  representation  and  logical  form,”  TR  367,  Computer 
Science  Dept,  U.  Rochester,  January  1991. 

Poesio,  M„  “Relational  semantics  and  scope  ambiguity,”  in  J.  Barwise,  J.M.  Gawron,  G. 
Plotkin,  and  S.  Tutiya  (Eds.).  Proc.,  1st  Conf.  on  Situation  Theory  and  its  Applications  (ST ASS 
90,  Loch  Crannoch,  Scotland).  CSLI,  September  1990. 

Poesio,  M.,  “Dialog-oriented  ABoxing,”  in  Z.  Ras,  M.  Zeraankova,  and  M.  Emrich  (Eds.). 
Proc.,  Int’L  Symp.  on  Methodologies  for  Intelligent  Systems  (ISMIS  90,  Knoxville,  TN,  October 
1990).  North-Holland,  277-288,  1990. 
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Schubert,  L.K.,  “Monotonic  solution  of  the  frame  problem  in  the  situation  calculus:  An  efficient 
method  for  worlds  with  fully  specified  actions,”  TR  306,  Computer  Science  Dept,  U.  Rochester, 
August  1989;  also  in  H.E.  Kyburg,  Jr.,  R.P.  Loui,  and  G.N.  Carlson  (Eds.).  Knowledge 
Representation  and  Defeasible  Reasoning.  Dortrecht:  Kluwer  Academic  Publishers,  23-67, 1990. 

Schubert,  L.K.,  “Semantic  nets  arc  in  the  eye  of  the  beholder,”  TR  346,  Computer  Science  Dept., 
U.  Rochester,  May  1990;  in  J.  Sowa  (Ed.)  Principles  of  Semantic  Networks:  Explorations  in  the 
Representation  of  Knowledge.  San  Mateo,  CA:  Morgan  Kaufmann,  Publishers,  95-107, 1991. 

Schubert,  L.K.  and  C.H.  Hwang,  “An  episodic  knowledge  representation  for  narrative  texts,” 
Proc.,  1st  lnt'l.  Conf.  on  Principles  of  Knowledge  Representation  and  Reasoning ,  Toronto,  May 
1989;  TR  345,  Computer  Science  Dept,  U.  Rochester,  May  1990. 

Schubert,  L.K.  and  C.H.  Hwang,  “Picking  reference  events  from  tense  trees:  A  formal, 
implementable  theory  of  English  tense-aspect  semantics,”  Proc.,  DARPA  Speech  and  Natural 
Language  Workshop,  Hidden  Valley,  PA,  34-41,  June  1990. 

Small,  S.L.,  “Parsing  as  cooperative  distributed  inference:  Understanding  through  memory 
interactions,”  in  M.  King  (Ed.)  Parsing  Natural  Language.  London:  Academic  Press,  1983. 

Small,  S.L.,  “The  story  in  mind  and  matter,”  commentary  on  article  by  R.  Wilensky,  Behavioral 
and  Brain  Sciences,  1983. 


4.  Computational  Learning  Theory 

The  work  in  this  area  concerned  the  formal  underpinnings  of  language  learning.  In  particular, 
we  have  been  concerned  with  developing  definitions  of  language  learning  that  are  reasonable  for 
natural  language,  as  opposed  to  unintuitive  artificial  languages  often  used  in  the  literature.  This 
work  may  lead  to  hints  about  useful  techniques  for  learning  concrete  problems.  In  addition,  the 
work  of  Sanjay  Jain  investigated  the  effects  of  faulty  data  and  extra  information  on  the  leamability 
of  classes  of  languages. 

Fulk,  M.A.  and  S.  Jain,  “Learning  in  the  presence  of  inaccurate  information,”  TR  279,  Computer 
Science  Dept.,  U.  Rochester,  March  1989. 

Fulk,  M.A.,  “Robust  separations  in  inductive  inference,”  presented,  Computational  Learning 
Theory  (COLT  90),  Rochester,  NY,  August  1990;  1990-91  Computer  Science  and  Engineering 
Research  Review,  Computer  Science  Dept,  U.  Rochester,  26-30,  September  1990;  Proc.,  31st 
Foundations  of  Computer  Science  (FOCS),  St  Louis,  MO,  October  1990. 

Fulk,  M.A.  and  S.  Jain,  “Open  problems  in  Systems  that  Learn,”  TR  285,  Computer  Science 
Dept,  U.  Rochester,  April  1989. 

Fulk,  M.A.  and  S.  Jain,  “Approximate  inference  and  scientific  method,”  TR  313,  Computer 
Science  Dept,  U.  Rochester,  October  1989. 
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5.  Representation  of  Plans  and  Action 


One  of  the  crucial  underlying  representation  problems  faced  in  this  project  was  the 
representation  of  actions  and  plans.  The  following  papers  addressed  this  issue  and  developed  new 
representations  of  actions  and  plans  that  are  more  powerful  than  developed  previously,  and  much 
better  suited  to  representing  communicative  plans  and  speech  acts. 


Allen,  J.F.,  ‘Towards  a  general  theory  of  action  and  time,”  Artificial  Intelligence  23, 2,  My  1984. 

Martin,  N.G.  and  J.F.  Allen,  “Abstraction  in  planning:  A  probabilistic  approach,”  presented. 
Workshop,  AAAI  Conf.  on  Artificial  Intelligence,  Boston,  MA,  My  1990. 

Martin,  N.G.  and  J.F.  Allen,  “Combining  reactive  and  strategic  planning  through  decomposition 
abstraction,”  DARPA  Workshop  on  Innovative  Approaches  to  Planning,  Scheduling,  and  Control, 
San  Diego,  CA,  November  1990. 

Tenenberg,  J.D.,  “Inheritance  in  automated  planning,”  Proc.,  1st  Int'l.  Conf.  on  Principles  of 
Knowledge  Representation  and  Reasoning,  Toronto,  May  1989. 

Tenenberg,  J.D.,  “The  robot  designer's  dilemma,”  1st  Int'l.  Workshop  on  Human  and  Machine 
Cognition,  Pensacola,  FL,  May  1989. 

Traum,  D.R.  and  J.F.  Allen,  “Causative  forces  in  multi-agent  planning,”  Proc.,  2nd  European 
Workshop  on  Modeling  Autonomous  Agents  and  Multi-Agent  Worlds  (MAAMAW90),  135-139, 
France,  August  1990. 

6.  Other  Research  in  Knowledge  Representation  &  Models  of  Parallel  Processing 
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